Automated testing scope (RSMS suite)
This document lists functionality, design decisions, and risk areas that merit automated tests, grounded in the current codebase and repository documentation. It is a coverage checklist, not a test plan with cases or owners.
Related docs: rsms-api-fastapi/docs/design-decisions.md, rsms-api-fastapi/docs/auth-planning.md, rsms-api-fastapi/docs/auth-implementation.md, rsms-results-storage-design, rsms-engine-data-flow, rsms-worker-implementation-plan, rsms-mass-balance-legacy-parity, rsms-master-tasklist.
1. Goals and test layers
| Layer | Typical scope | Suites in this repo today |
|---|---|---|
| Unit | Pure helpers (IDs, domain parsing, JSON rounding, bag merge logic) | rsms-api-fastapi/tests/, rsms-worker-function/tests/ (partial) |
| Service / API integration | FastAPI TestClient + dependency overrides or fakes for Azure | Sparse |
| Worker integration | Queue message → local run folder (optional; often slow / needs EXEs) | Minimal |
| E2E | Browser + real or dockerized backends | None for React |
Prioritize unit + API integration for the FastAPI service; add contract tests between API and worker payloads.
2. RSMS API (FastAPI) — authentication and sessions
Documented in rsms-api-fastapi/docs/auth-implementation.md and rsms-api-fastapi/docs/auth-planning.md.
| Topic | Why test | Notes |
|---|---|---|
| Login | Core entry; password verify + JWT shape | Wrong password → 401; success → access + refresh tokens |
| Refresh | SPA longevity | Valid refresh → new access; invalid / wrong type → 401 |
| Access JWT | All protected routes | Missing / malformed / expired → 401 with WWW-Authenticate |
| Token payload | Security | sub must be UUID; type: "access" enforced; user must exist in store |
GET /auth/me | Client bootstrap | Returns user without password; 401 if not logged in |
| Logout | Contract | If implemented as stateless, document behavior; any cookie clearing |
| Password hashing | Regression | bcrypt verify path; do not store plaintext |
| JWT settings | Ops | JWT_ACCESS_TOKEN_EXPIRE_MINUTES, JWT_REFRESH_TOKEN_EXPIRE_DAYS, algorithm HS256 |
3. RSMS API — role-based authorization (RBAC)
Matrix from rsms-api-fastapi/docs/auth-planning.md and rsms-api-fastapi/docs/auth-implementation.md.
| Area | super_admin | analyst | viewer |
|---|---|---|---|
User CRUD (/users) | Allow | 403 | 403 |
| POST/PUT/DELETE riverbasin | Allow | 403 | 403 |
| GET riverbasin | Allow | Allow | Allow |
| POST/PUT/DELETE scenario | Allow | Allow | 403 |
| GET scenario / list | Allow | Allow | Allow (subject to domain rules below) |
| POST …/run (enqueue) | Allow | Allow | 403 |
| GET …/engine-files.zip | Allow | Allow | 403 (if unchanged) |
Automated checks: fixed user fixtures per role + TestClient; assert exact HTTP status (401 vs 403).
4. RSMS API — scenario author domain (tenant-style gate)
Implementations: app/auth/scenario_domain.py, RSMS_SCENARIO_DOMAIN_ENFORCEMENT in config, guards in app/main.py and app/results/service.py.
| Topic | Why test |
|---|---|
| Domain extraction | Normalization; invalid emails |
| Same domain vs other domain | Core policy |
super_admin bypass | Must not regress |
Legacy / non-email author | Locked out for non-admin |
Toggle RSMS_SCENARIO_DOMAIN_ENFORCEMENT=false | Dev / migration |
| Results routes | Same as scenario read |
| 404 vs 403 | Product choice |
5. RSMS API — scenario author immutability
Design: author is server-set on create; not on ScenarioCreate / ScenarioUpdate models.
| Topic | Why test |
|---|---|
PUT body must not change author | Even if clients send extra JSON |
| POST create | author in response matches authenticated user email |
6. RSMS API — IDs and entity hierarchy
Docs: rsms-api-fastapi/docs/design-decisions.md (note: prose still mentions UUIDs in places; implementation uses R-YYMMDD-xxx / S-YYMMDD-xxx in app/ids.py).
| Topic | Why test |
|---|---|
normalize_riverbasin_id / normalize_scenario_id | Regex, casing, whitespace |
| HTTP mapping | User-facing errors |
next_*_id | Collision avoidance |
| Scenario partition key | Data model |
| Run + zip: basin match | Safety |
7. RSMS API — Azure Table / blob behavior (with fakes)
| Topic | Why test |
|---|---|
| In-memory fallback | Dev without Azure |
| Scenario JSON bags | Regression |
| Status coercion | Legacy strings |
spill_date_epoch vs scenario_created_at_epoch | Field semantics |
| Riverbasin create | Size limits, 503 |
| Delete riverbasin cascade | Data loss path |
| Delete scenario | Results cleanup |
8. RSMS API — worker queue and engine artifacts
| Topic | Why test |
|---|---|
| Enqueue payload | Contract with worker |
| 503 when queue not configured | Ops |
202 + status Queued | UX |
| Engine zip | Empty vs zip stream |
9. RSMS API — results read path
Docs: rsms-results-storage-design (§2–4, JSON rounding).
| Topic | Why test |
|---|---|
| Plume table decode | Binary float32 layouts |
| Concentration JSON rounding | Stable API |
| Mile / hour axis rounding | Avoid float tails |
| Missing scenario / wrong basin | Errors |
| Mass balance + edge JSON | Blob read path |
| RMI / river-mile axis | Geometry |
RESULTS_API_ENABLED | Feature flag |
10. RSMS worker (rsms-worker-function)
Docs: rsms-engine-data-flow, rsms-worker-ohio-riverflows-tls.
| Topic | Why test |
|---|---|
| Queue message parsing | Contract |
| ID helpers parity with API | Single contract |
| RMI/blob helpers | Input validation |
nfq_from_api / RiverFlows client | Production fragility |
| Results write path | SSOT alignment |
| Engine workflow (full) | Expensive |
11. RiverFlows API (Django) — if treated as part of the suite
Not required for “RSMS API only,” but worker depends on it.
| Topic | Why test |
|---|---|
multi-riverflows-duration (or equivalent) | Worker NFQ merge |
| Auth | If any |
| TLS / host | Align with rsms-worker-ohio-riverflows-tls |
12. Frontend (rsms-frontend-react)
No automated tests are present today; high-value targets:
| Topic | Why test | Suggested tool |
|---|---|---|
| Auth: token attach, 401 → login | Core UX | Vitest + MSW, or Playwright |
| Role-gated UI | Match RBAC | Component tests: hide run/create for viewer |
| Scenario list / detail | Domain errors | Mock API 404 for foreign scenario |
| Results charts | Regression | Visual or data snapshot with fixed API fixtures |
| Routing | Deep links | E2E: ScenariosPage / ResultsPage URL-driven state (see in-file comments in rsms-frontend-react/src/pages/ScenariosPage.jsx) |
13. Cross-cutting and non-functional
| Topic | Why test |
|---|---|
| CORS configuration | SPA + API split |
| Trusted hosts | Production hardening |
| Logging / health | Ops |
| Mass balance parity | Science trust |
14. Suggested priorities (pragmatic)
- P0 — API: RBAC matrix + domain access + results guard (integration tests with auth overrides and fake storage).
- P0 — API: ID normalization and “scenario not in basin” on run/zip.
- P1 — API: Scenario bag merge / status coercion; enqueue +
Queuedupdate. - P1 — Worker: Queue payload + ID parity tests run in CI on every PR.
- P2 — Results: Rounding and tolerance behavior (table-driven unit tests from design doc).
- P2 — Frontend: Smoke E2E login + one scenario flow.
- P3 — Full engine: Nightly or manual pipeline with real EXEs.
15. Maintenance
When design docs change (storage keys, roles, domain policy, ID format), update this document and add or adjust tests in the same change. When OpenAPI changes, regenerate or refresh client fixtures used in integration tests.